|
|
|
| e-Pub |
Previous | 
Home
Section: New Results
Pedestrian Detection using Convolutional Neural Networks
Participants : Ujjwal Ujjwal, François Brémond.
Keywords: Pedestrian detection, CNN
The objective of the work was to perform pedestrian detection in different settings. The settings corresponded to different types of camera-views as well as different types of camera settings (e.g- moving camera vs. static camera). The work followed a wide range of experiments using different public implementations of convolutional neural networks and on different types of datasets. We detail the experiments one by one in the following subsections :
Experiments on CNN architectures
We started with an evaluation of different CNN architectures for pedestrian detection. Towards this end, we implemented three important and famous architectures - LeNet [72] , AlexNet [69] and CifarNet [68] . For the purpose of training and validation we extracted patches from the public datasets of Inria [55] , Daimler [58] , TUD-Brussels [92] , Caltech [57] , ViPer [62] , USC [93] and MIT [78] . The breakup of the dataset used for training was as shown in table 1 .
| Pedestrian Patches | Non-Pedestrian Patches | |
| Training | 131,183 | 61,500 |
| Validation | 65,591 | 30,700 |
| Testing | 65,591 | 30,600 |
Implementation of all the three models for pedestrian detection which gave a very high accuracy ( (LeNet), (AlexNet) and (CifarNet) ) for classification at patch level. Though these results were good at the patch level, more thorough understanding was needed to determine the effect of network architecture on classification. This was important because the three architectures vary greatly in terms of number of layers and other parameters. Moreover the practical problem in pedestrian detection chiefly involves detecting pedestrians in an image (i.e. when full-scale images instead of pre-defined patches are available).
The first set of experiments was done using sliding windows. This had to be abandoned soon, since for each image this was taking an impractical time (). Moreover sliding window is less suited in its naive setting due to the fact that each candidate window had to be rescaled to meet the network input size and tested individually by extracting features over it. This was followed by efforts to understand and implement a wide range of other techniques for full-scale detection using CNN. This is still an open problem though some encouraging advancements through R-CNN [61] and OverFeat [83] have been made. A major difficulty lies in lack of robust implementations of CNN which allow for integrated training and testing with object localization. Moreover existing implementations are less flexible and often make it difficult to carry out modifications required to implement new techniques independently.
We settled with the R-CNN which uses region proposals extracted using selective search [87] to extract object proposals and then train a CNN using those proposals and subsequently classifying using a SoftMax classifier or a SVM.
The evaluation was done on both moving cameras and static cameras and the evaluation showed that the network was performing a little satisfactorily, though below the state-of-art performance standards. The performance metric was Average Miss Rate (AMR) Vs. False Positives Per Image (FP/I). A good detector must exhibit a very low AMR alongwith very low FP/I. Table 2 summarizes the detection results, with table 3 summarizing state-of-art results on different pedestrian detection datasets.
| Dataset | #Images | AMR | FP/I |
| Inria | 741 | 0.27 | 0.36 |
| DAIMLER | 15K | 0.38 | 0.40 |
| Caltech | 16K | 0.46 | 0.43 |
| USC | 584 | 0.02 | 0 |
| PETS 2009 S2.L1 | 5565 | 0.42 | 0.29 |
| PETS 2009 S2.L2 | 1744 | 0.35 | 0.19 |
| Dataset | #Images | AMR | FP/I |
| Inria | 741 | 0.14 | 0.1 |
| DAIMLER | 15K | 0.29 | 0.21 |
| Caltech | 16K | 0.12 | 0.1 |
| PETS 2009 S2.L1 | 5565 | 0.22 | 0.1 |
| PETS 2009 S2.L2 | 1744 |
It was felt that more work is needed to organize a CNN library and subsequently work to improve the above results.
Conclusion
Pedestrian detection finds its applications in different settings. It is also highly influenced by a wide variety of variations which have many practical ramifications in areas such as surveillance. It is important to develop a robust and high-performance system for pedestrian detection that is able to take into account a very wide range of such variations such as occlusion and poor visibility. CNNs have shown great promise in object detection and recognition lately and this inspires it growing applications in pedestrian detection. While the current results of our R-CNN experiments do not match the state-of-art it has shown some promise by providing consistent numbers across datasets which shows that CNNs are a good way to transcend a system beyond dataset-specific restrictions. An important factor is the instance of moving cameras vis-a-vis static cameras. While the present experiments show that decent performance is obtained on moving camera databases, consistent and similar performance is also obtained in the context of static camera databases such as PETS. This shows that with better training and improved practices of dataset handling such as augmentation and dataset structuring by clustering based methods can help in pushing the performance to acceptable levels for applications in automated surveillance and driving applications.
Further Work We intend to take this study forward, by looking into novel approaches to gather more information about a pedestrian dataset from CNN, while further increasing the detection results.